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O 

The goal of this note is to explain a problem with the proof of Nash's theorem in [5], 

which is the main document attached below. We will use the notation of that paper through- 

_ out without further comment, with the exception that each document has its own section 

numbers and list of references. We are indebted to Sergiu Hart and Eran Shmaya for their 

careful study of [5] which led to their simultaneous discovery of this error. So far the error 

has not been fixed, but many of the results and techniques of the paper remain valid, so we 

will continue to make it available online. 

H 

P 1 The problem 

The error is in our proof of Lemma 4.6, which is the main driver behind our assertion that 

^ order m exchangeable equilibria exist (Theorem 4.7). Of course they do exist; this is the main 

t> benefit of setting out to prove something you already know to be true. To our knowledge 

everything before and after this step remains unaffected, but this is an important step in the 

middle of the proof and logically it must be patched if we are to reach the desired conclusion. 

The particular problem is our implicit use of the "fact" below. In the incorrect proof of 

Lemma 4.6 of [5], this "fact" was applied with H"T in place of T and appropriate choices of 

* G Ag x5m (H"T) and r G Ag x ^(ITT). 

"Fact" 1. Let a,r G A n (T) be two strategy profiles such that C{ is payoff' equivalent to Tj in 
r for all i. Then o and r are payoff equivalent when viewed as strategies for the maximizer 
in the associated zero-sum game T°. 

This may sound reasonable at first because it seems that the difference between payoff- 
equivalent strategies should not be game-theoretically significant. In fact it is false! And not 
just for odd counterexamples: it is false generically. 

Example 2. To see how this can fail, let T be the following symmetric bimatrix game: 



A = B ri 
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Let player i's strategy set be Ci = {a,i,bi,Ci}. The first two columns of A are equal, as are 
the first two rows, so Oi is payoff equivalent to 61. Symmetrically, a 2 is payoff equivalent to 
b 2 . In the associated zero-sum game T we have 

u° M ((ai, a 2 ), (01, ci)) = 1^0 = 4((i)i, 6 2 ), (ai, ci)), 

so (01,02) and (61,62) are not payoff equivalent in T . 

Why does this happen? The reason is that when we consider correlated strategies, we are 
interested not only in what utility our recommendations give us, but also in what information 
they give. While we may not care from a payoff perspective whether strategy Oj or Oj has 
been suggested, the two possibilities may allow us to make completely different inferences 
about the behavior of our opponents. It is this information which may be payoff- relevant. 

To see this in action, consider the following two distributions over outcomes of T: 
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One can verify that W l is a correlated equilibrium, and W 2 is obtained from W l by mov- 
ing some mass from (0^02) and (ci,a 2 ) to (b±, c 2 ) and (ci,6 2 ), which are profiles of payoff 
equivalent strategies. Nonetheless, W 2 is not a correlated equilibrium. If player 1 receives 
the recommendation 61 then he knows with probability one that player 2 has received the 
recommendation c 2 , hence it is in the interest of player 1 to deviate to c\. 



2 A digression on continuous games 

In this section we highlight the complications discussed in Section 1 by looking at how 
they manifest themselves in the geometry of Nash and correlated equilibria of continuous 
games 1 . Existence of both types of equilibria in continuous games is generally proven in four 
steps: 1) discretize the strategy spaces to obtain a finite game, 2) prove the corresponding 
existence theorem for finite games, 3) observe that an exact equilibrium of the finite game 
is an approximate equilibrium of the continuous game, and 4) show that a limit of such 
approximate equilibria is an exact equilibrium of the continuous game. 

Steps 1 and 3 are more or less content-free. Step 2 can be accomplished for Nash equilibria 
by fixed-point methods or for correlated equilibria by Hart and Schmeidler's clever minimax 
argument. There is an interesting divergence at step 4: in the case of Nash equilibria it is 
trivial and in the case of correlated equilibria it seems to require a hairy measure-theoretic 
argument (see [1] or [6]). 

The difference is easiest to understand in terms of an auxiliary map e : A(r) —¥ M>o- 
This is defined by the condition that e(/i) is the smallest e such that \x is an e-correlated 
equilibrium (no measurable deviation yields an expected gain of more than e). Let e 11 be 



■"■For concreteness' sake, define these to be games with finitely many players, compact metric strategy 
spaces, and continuous utility functions. 



the restriction of e to A n (T). Then e n (/z) is also the smallest e such that // is an e-Nash 
equilibrium in the usual sense. 

We can study the map e via two different topologies on its domain A(r): the weak and 
strong topologies. Two measures are near each other in the strong topology if you can get 
from one to the other without moving much mass. They are near each other in the weak 
topology if you can get from one to the other without moving much mass very far. 

It is easy to show that e is continuous with respect to the strong topology, but this is not 
very useful because A(r) is not compact with respect to this topology (at least when T is 
not a finite game) so the limiting arguments do not go through. It is compact with respect 
to the weak topology and viewing A n (T) as a space of tuples it is easy to show that e 11 is 
continuous with respect to the weak topology, hence we get the necessary limiting argument 
for the existence of Nash equilibria. 

For correlated equilibria the situation is different: e is not weakly continuous! The 
argument is essentially the same as why W 1 was a correlated equilibrium in Example 2 but 
W 2 was not. We can move some mass a little bit (thinking now of strategically equivalent 
strategies as being quite "close" to each other) and change the information conveyed by the 
correlating device drastically. This changes an exact correlated equilibrium to a distribution 
for which e is bounded away from zero, no matter how small the distance we moved the 
mass. 

In particular this shows that e is not weakly upper semicontinuous. In fact this doesn't 
matter; all we really need for the existence proof is weak lower semicontinuity. It turns out 
e has this property, but proving it is a delicate technical matter (again, see [1] or [6]). 

The fact that Nash equilibria behave nicely with respect to strategic equivalence is what 
allows us to turn problems regarding mixed Nash equilibria of polynomial games into finite- 
dimensional problems in moment spaces: see for example [2] and [3] . The failure of correlated 
equilibria to behave nicely with respect to strategic equivalence is captured by the fact that 
the corresponding problems for correlated equilibria are inherently infinite-dimensional [4] . 

3 A more specific problem 

The erroneous proof of Lemma 4.6 claims to construct, for any symmetric mixed strategy y 
of the minimizer in the game (S m r)°, a mixed strategy ir y e AG X s m (n m r) for the maximizer 
such that u° M (7T y ,y) = 0. Furthermore, the construction implicitly claims that such a n y can 
be found which does not depend on the utilities of T (as is the case in [1]) and which is 
rational in y. That is to say, the elements of 7r y are in the field generated by the elements of 
y over Q. For simplicity we will write ii y G Q[y]. 

Let us see how rationality of n y follows from the proof of Lemma 4.6. First note that a 
finite zero-sum game has a maximin strategy which is rational in the utilities, because the 
set of maximin strategies is a polytope defined by inequalities with coefficients linear in the 
utilities. Therefore Lemmas 2.15 and 3.8 show that the certificates ir y constructed in Hart 
and Schmeidler's proof of the existence of correlated equilibria and our proof of the existence 
of exchangeable equilibria are in Q[y]. But Lemma 4.6 claims to construct the certificate 



n y for order m exchangeable equilibria by taking the one from Lemma 3.8 for exchangeable 
equilibria and replacing it by the product of its marginals. That is to say, we are taking the 
image of a tuple of elements of Q[y] under a polynomial map, and so we end up with another 
tuple ir y eQ[y]. 

Example 3. In fact such a n y need not exist for order m exchangeable equilibria, as the 
following example due to Sergiu Hart shows. Consider the symmetric bimatrix game T and 
its contracted second power S 2 r shown in Table 1. Suppose the minimizer plays the mixed 
strategy y a ^ which assigns probability a to 00 — > 01 and 00 — > 10 and probability (3 to 
11 — >■ 01 and 11 — > 10 for each player, so we have 4(a + f3) = 1. 

To have 7i ya <c e A^ 2X s 2 {H 2 T) means that -n Va ^ is a product of four i.i.d. Bernoulli^) 
random variables for some p G [0, 1]. For notational simplicity we will let q — 1 — p. Then 
we can compute the expected utility of the maximizer in (S 2 r)° as: 

u° M (iT y ^,y a ^) = 4(aq 2 - f3p 2 )(qr + ps). 

For this expression to be zero for all r and s we must have aq 2 = (3p 2 , so 

p = —= -= and q = —— — , 

V« + y/P y/a + \/P 

and these expressions are clearly not rational in a and /3. But p and q are marginals of -K Va <P 
and so can be written as sums of its elements. Thus the entries of 7r J/a '' 3 cannot all be rational 
in a and (3 either. 
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A new proof of Nash's Theorem via 
exchangeable equilibria 

Noah D. Stein, Pablo A. Parrilo, and Asuman Ozdaglar* 

September 14, 2010 

o 

(N 

^ Abstract 

We give a novel proof of the existence of Nash equilibria in all finite games without 
using fixed point theorems or path following arguments. Our approach relies on a 
new notion intermediate between Nash and correlated equilibria called exchangeable 
equilibria, which are correlated equilibria with certain symmetry and factorization 
properties. We prove these exist by a duality argument, using Hart and Schmeidler's 
proof of correlated equilibrium existence as a first step. 

In an appropriate limit exchangeable equilibria converge to the convex hull of Nash 
equilibria, proving that these exist as well. Exchangeable equilibria are defined in 

£T} terms of symmetries of the game, so this method automatically proves the stronger 

statement that a symmetric game has a symmetric Nash equilibrium. The case without 

,£J symmetries follows by a symmetrization argument. 

o 
m 

in 1 Introduction 

o 

Nash's Theorem is one of the most fundamental results in game theory and states that 
any finite game has a Nash equilibrium in mixed strategies. Despite its importance, the 
authors of the present paper know of only three essentially different proofs. The first and 
most common way to prove Nash's Theorem is by applying a fixed point theorem, usually 
Brouwer's or Kakutani's, to a map whose fixed points are easily shown to be Nash equilibria. 
The fixed point theorem is usually proven combinatorially, say by Sperner's Lemma [15] or 
Gale's argument using the game of hex [5], or with (co-)homology theory, a suite of powerful 
but less elementary tools from algebraic topology [8]. 

The second proof of Nash's Theorem (historically) is algorithmic and consists of showing 
that the Lemke-Howson path-following algorithm terminates at a Nash equilibrium [11]. In 
fact this is not so different from the fixed point proof, because Sperner's Lemma is also 
proven by a path- following argument. 
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The third proof of Nash's theorem is due to Kohlberg and Mertens and is topological 
[10] . The idea is to simultaneously consider the set of all games of a given size and the set of 
all (game, equilibrium) pairs. Under appropriate compactifications both of these sets become 
spheres of the same dimension. One then shows that the mapping sending an equilibrium to 
the corresponding game is homotopic to the identity map on the sphere. A (co-)homological 
or degree-theoretic argument shows that such a map must be onto [8]. Technically speaking 
this step is almost identical to the proof of Brouwer's Fixed Point Theorem, so the third 
proof is closely related to the first. 

All three proofs have provided unique insights into the structure of Nash equilibria and it 
is our hope that a different proof, which uses neither fixed point theorems, nor path-following 
arguments, nor any algebro-topological tools, will provide further insights. 

Hart and Schmeidler [7] have proven the weaker result that correlated equilibria exist by a 
clever application of the Minimax Theorem, summarized in Section 2.2. For games endowed 
with a group action, a simple averaging argument then proves that a symmetric correlated 
equilibrium exists (Proposition 2.26). We show that for such games Hart and Schmeidler's 
proof can be strengthened to produce correlated equilibria with additional symmetry and 
factorization properties, which we call exchangeable equilibria (Theorem 3.9). 

To illustrate this idea, consider the case of k x k symmetric bimatrix games (two-player 
games fixed under the operation of swapping the players). Let X = {xx T \ x G K^ 1 }. Then 
we have 

Nash convex hull of Nash exchangeable correlated 

CEHX C conv(CE n X) C CE n conv(X) C CE , 

where each type of (symmetric) equilibrium is defined by the set written below it. This 
definition shows that the exchangeable equilibria are a natural mathematical object. For 
examples and game-theoretic interpretations of exchangeable equilibria, see the companion 
paper [16]. 

In Section 3 we extend the definition of exchangeable equilibria games to n-player games 
with arbitrary symmetry groups. The preceding discussion shows that the set of exchange- 
able equilibria is convex, compact, contained in the set of symmetric correlated equilibria, 
and contains the convex hull of the set of symmetric Nash equilibria. One can show that 
these containments can all be strict [16] , so proving existence of exchangeable equilibria is a 
step in the right direction, but does not immediately prove existence of Nash equilibria. 

However, we can use the same techniques to prove existence of exchangeable equilibria 
with successively stronger symmetry properties as follows. For a fixed n-player game V and 
number m G N, we define two new games n"T and H"T, which we call m th powers of T. 
These are larger games in which m copies of T are played simultaneously. The difference 
between the two powers is that IPT has a different group of players for each copy, so mn 
players total, whereas S"T has just one group of n players playing all m copies, but perhaps 
choosing different actions in each copy (Figure 1). 

There is a natural marginalization map sending exchangeable equilibria of either of these 
powers to exchangeable equilibria of T. In fact, any exchangeable equilibrium of T can be 
lifted to an exchangeable equilibrium of either power, and for a symmetric Nash equilibrium 
we can take these two lifts to be the same. However, for a general exchangeable equilibrium 



the two lifts need not be compatible, so it is natural to consider the intersection XE m (T) : = 
XE(n m r) flXE(5 m T) of the sets of exchangeable equilibria of the two powers. We call these 
order m exchangeable equilibria of F and prove they exist using a similar Minimax argument 
(Theorem 4.7). 

Under appropriate identifications these sets turn out to be nested as m grows and, being 
convex, they all contain the convex hull of the symmetric Nash equilibria: 

NE(r) C conv(NE(r)) C . . . C XE 3 (T) C XE 2 (T) C XE x (r) = XE(T) C CE(r). 

Assuming V has a rich enough symmetry group (e.g. if it is a symmetric bimatrix game or, 
more generally, if it satisfies a condition we call player transitivity), then as m goes to infinity 
these converge to symmetric correlated equilibria in which the outcome of the correlating 
device is common knowledge; such correlated equilibria are known to be mixtures of sym- 
metric Nash equilibria (Theorems 4.9 and 5.1). In particular, this proves that symmetric 
Nash equilibria exist in games with rich enough symmetry groups. 

Note that symmetry is fundamental in this argument. For example, if we had begun with 
a general bimatrix game and let X = {xy T \ x,y G IR^q 1 } we would have had conv(X) = 
R>q , so the exchangeable equilibria (even the order m exchangeable equilibria) would have 
been exactly the correlated equilibria and we would not have strengthened the equilibrium 
notion at all. However, there are several ways of turning general games into symmetric games 
[6] and applying such a procedure proves existence of Nash equilibria in all games (Section 
5.2). 

Up to the step of taking m to infinity, all the steps of our proof are computationally effec- 
tive. Papadimitriou has shown how to apply the ellipsoid algorithm to Hart and Schmeidler's 
proof to efficiently compute a correlated equilibrium of a large game [13]. The same tech- 
nique applied to our proof allows one to compute an exchangeable equilibrium (or an order 
m exchangeable equilibrium for fixed m) in polynomial time, even though the set of these 
is not polyhedral. Computing these is interesting in its own right [16] and may be useful 
for computing approximate Nash equilibria. However, computation is not the focus of this 
paper and we leave a detailed investigation of these ideas for future work. 

The remainder of the paper is organized as follows. We begin with background material 
in Section 2. We cover the definitions of games and equilibria, give an overview of Hart 
and Schmeidler's proof of the existence of correlated equilibria so we can modify it later, 
and introduce group actions. In Section 3 we introduce exchangeable equilibria and prove 
existence of these for games under arbitrary group actions. We do the same for order m ex- 
changeable equilibria in Section 4, introducing powers of games along the way. We complete 
the argument in Section 5 by showing that the order m exchangeable equilibria converge to 
mixtures of symmetric Nash equilibria in games with a player transitive symmetry group, 
and then showing that we can symmetrize any game to make this condition hold. Section 6 
concludes and gives directions for future work. 



2 Background 

This section is divided into three parts. In the first part we lay out the basic definitions of 
finite games as well as Nash and correlated equilibria to fix notation. We assume the reader 
is familiar with these concepts and do not attempt to motivate them. The second part 
reviews Hart and Schmeidler's proof of the existence of correlated equilibria [7], preparing 
for similar arguments in this paper. The third part covers symmetries of games. 

The concept of a symmetry of a game extends back at least to Nash's paper [12]. Sym- 
metries are fundamental to the present paper, so we spend more time on these and give 
some examples. Although we use the language of group theory to discuss symmetries, it is 
worth noting that we do not use any but the most basic theorems from group theory (e.g., 
the fact that for any h in a group G, the maps g y-¥ gh and g i— >■ hg are bijections from G 
to G). Everything in this section is standard except for the notions of a good reply, a good 
set, a player-trivial symmetry group, a player-transitive symmetry group, and the remarks 
following the statement of Nash's Theorem. 

2.1 Games and equilibria 

Definition 2.1. A (finite) game has a finite set / of n > 2 players, each with a finite set Ci 
of at least two strategies (also called pure strategies) and a utility function Ui : C — > R, 
where C = Y[ Cj- A game is zero-sum if it has two players, called the maximizer (denoted 
M) and the minimizer (denoted m), and satisfies um + u m = 0. 

For elements of Cj we use Roman letters subscripted with the player's identity, such as 
Si and tj. We will typically use the unsubscripted letter s to denote a strategy profile (a 
choice of strategy for each player) . For a choice of a strategy for all players except % we use 
the symbol s_j. To denote the set of Borel probability distributions on a space X we write 
A(X). For much of the paper X will be finite so we can view A(X) as a simplex in the 
finite-dimensional vector space R x of real- valued functions on X. For x G X the probability 
distribution which assigns unit mass to x will be written S x G A (A). 

Definition 2.2. A mixed strategy for player % in a game T is a probability distribution 
over his pure strategy set C iy and the set of mixed strategies for player i is A(Cj). The set 
of mixed strategy profiles (also called independent or product distributions) will be 

denoted A n (r) :=n i A(C J )- 

For independent distributions it is important that we write A n (r) rather than A n (C), 
because T specifies how C is to be thought of as a product. For example, the set S x S x S 
could be viewed as a product of three copies of S, or a product of S with S x S, and these 
lead to different notions of an independent distribution - one is a product of three terms 
and one is a product of two terms. This distinction will be particularly important when we 
define powers of games in Section 4. 

To make the notation fit together we will write A(r) for A(C). We may then view A n (r) 
as the (nonconvex) subset of A(r) consisting of product distributions or as a convex subset 



of R UiC \ The former view will be natural when we define exchangeable equilibria, which 
live in A(T), as convex combinations of product distributions. The latter will be useful 
when looking for product distributions which are fixed by a group action (see the proof of 
Lemma 3.8); such fixed distributions are easy to find with a convex setup (Proposition 2.18). 
Which of these views we are using will be clear from context if not explicitly specified. 

As usual we extend the domain of Ui from C to A(r) by linearity, defining u^ti) = 
^2 seC Ui(s)n(s). Having done so we can define equilibria. 

Definition 2.3. A Nash equilibrium is an n-tuple (pi,...,p n ) G A n (r) = f|jA(Cj) of 
mixed strategies, one for each player, such that Wj(sj,p_j) < Ui(pi, p-i) for all strategies 
Si G Ci and all players i. The set of Nash equilibria of a game V is denoted NE(r). 

Definition 2.4. A correlated equilibrium is a joint distribution 7r G A(r) such that 
X/s_-ec_- [ u i^ii s ~i) ~ u i( s )} n ( s ) < f° r & H strategies Si,ti G Ct and all players i. The set 
of correlated equilibria of a game T is denoted CE(r). 

The following alternative characterization of correlated equilibria is standard and we omit 
its proof. Suppose (X 1; . . . ,X n ) is a random vector taking values in C. We think of JQ as 
a (random) strategy recommended to player i. Given this information, player % can form 
his conditional beliefs P(X_j | Xj) about the recommendations to the other players given 
his own recommendation. That is to say, P(X_j | Xi) is a random variable taking values in 
A(C_j) which is a function of Xi. One can then define the event 

{the pure strategy Xi is a best response to the distribution P(X_j | Xi) for all i}. 

The distribution of (X\, . . . , X n ) is a correlated equilibrium if and only if this event happens 
almost surely. More succinctly: 

Proposition 2.5. Let (Xi, . . . ,X n ) be a random vector taking values in C distributed ac- 
cording to it G A(r). Then ix is a correlated equilibrium if and only if Xi is a best response 
to P(A_j | Xi) almost surely for all i. 

Nash equilibria correspond exactly to the correlated equilibria which are product distri- 
butions, so viewing A n (r) as a subset of A(r) we can write NE(r) = CE(r) fl A n (r). We 
introduce the existence theorems for correlated and Nash equilibria in Sections 2.2 and 2.3. 

We need the Minimax Theorem at this point to define the value of a zero-sum game. It 
also plays an important role in our proof of Nash's Theorem. An elementary proof can be 
given using the separating hyperplane theorem [2]. 

Minimax Theorem. Let U and V be finite- dimensional vector spaces with compact convex 
subsets K C U and L C V . Let <& : U X V — > M. be a bilinear map. Then 



sup inf $(x, y) = inf sup $(x, y), 

x&KV^L V^ L x&K 



and the optima are attained. 



Definition 2.6. Given a zero-sum game T, we can apply this theorem with K = A(Cm), 
L = A(C m ), and $ = Um- The common value of these two optimization problems is called 
the value of the game and denoted v(T). Maximizers on the left hand side are called 
maximin strategies and the set of such is denoted Mm(T) C A(Cm)- 

We now introduce the notion of a good reply in a zero-sum game. This is not a standard 
definition, but it will simplify the statements of several arguments below. The name is meant 
to be evocative of the term best reply: while a best reply is one which maximizes one's payoff, 
a good reply is merely one which returns a "good" payoff: at least the value of the game 1 . 

Definition 2.7. In a zero-sum game T, we say that a strategy a G A(Cm) for the maximizer 
is a good reply to 9 G A{C m ) if um(&, 9) > v(T). We say that a set E C A{Cm) of strategies 
is good against the set C A(C m ) if for all 9 G there is a a G E which is a good reply 
to 9. If E is good against A(C m ) we say that E is good. 

The main result about good sets is: 

Proposition 2.8. If V is a zero-sum game and E C A(Cm) is good, then T has a maximin 
strategy in conv(E) ; i.e., conv(E) D Mm(T) ^ 0. 

Proof. Apply the Minimax Theorem with K = conv(E) and L = A(C m ). □ 

It is worth noting that in general a good set need not include a maximin strategy. For 
example, in any zero-sum game the set Cm $1 A{Cm) is a good set, but some zero-sum 
games such as matching pennies only have mixed maximin strategies, i.e. Cm flMm(r) = 0. 

The notion of payoff equivalence is a standard way to turn structural information about 
a game into structural information about equilibria. The proofs of both propositions below 
are immediate. 

Definition 2.9. Two mixed strategies ctj,Tj G A(Cj) are said to be payoff" equivalent if 

Uj(ai, s_j) = Uj(Ti, s_j) for all s_j G C_i and all players j. 

Proposition 2.10. If T is a zero-sum game, E is good, and each a G E is payoff equivalent 
to some a' G E' ; then E' is good. 

Proposition 2.11. If ai is payoff equivalent to Ti for all i, then (<7i , . . . , o~ n ) is a Nash 
equilibrium if and only if (ri, . . . , r„) is a Nash equilibrium. 

2.2 Hart and Schmeidler's proof 

In this section we recall the structure of Hart and Schmeidler's proof of the existence of 
correlated equilibria based on the Minimax Theorem [7]. The goal of this is to frame their 
argument in a way which will allow us to extend it, redoing as little as possible of the work 
they have done. We will use similar arguments to prove Theorems 3.9 and 4.7. 



1 Good sets are similar in spirit to Voorneveld's prep sets [18], but tailored to zero-sum games. 



Hart and Schmeidler's argument begins by associating with a game T a new zero-sum 
game T and interpreting correlated equilibria of T as maximin strategies of this new game. In 
r° the maximizer plays the roles of all the players in T simultaneously and the minimizer tries 
to find a profitable unilateral deviation from the strategy profile selected by the maximizer. 

Definition 2.12. Given any game T, define a two-player zero-sum game T with C M := C, 
C^j := |J. Ci x d, and utilities 

7 .0 / / f \\ _ _ / / f \\ ._ J U i( S ) - «»(*«> S ~i) lf r 'i = S *> 

I (J otherwise. 

Proposition 2.13. Let F be any game. For any player i in V, n G Ci, and s G C we 
have u%(s,(ri,ri)) = 0, so we can bound the value of T° by v(T°) < 0. A mixed strategy 
a G A(C^) = A(C) for the maximizer in T° satisfies u M (o-, (rj,tj)) > for all {r^tj) G C^ 
if and only if a G CE(T). Therefore, if v(T°) = </ien Mm(r°) = CE(r). 

Proof. Immediate from the definitions. D 

To prove v(T°) = 0, and hence the existence of correlated equilibria (Theorem 2.16), 
we must show that for any y G A(C^J there is a n G A(C^) such that UM(^,y) > 0. 
Hart and Schmeidler actually show that there exists such a 7r with some extra structure, 
which we summarize in Lemma 2.15. We will exploit this extra structure below to prove 
stronger statements in a similar spirit: Lemmas 3.8 and 4.6. These in turn allow us to prove 
the existence of exchangeable equilibria (Theorem 3.9) and order m exchangeable equilibria 
(Theorem 4.7). 

Given a y = (yi, . . . ,y n ) G A(C^), t/j G M. ClXCi , a good reply n can be constructed in 
terms of certain auxiliary games 7(2/*). For the purposes of the present paper, it is more 
important to understand the statement of Lemma 2.15 than to remember the details of this 
construction. Besides this lemma the only property of 7(2/*) we will need is that its definition 
is independent of how elements of Ci are labeled (Proposition 3.7). 

Definition 2.14. For any player i in T and any nonnegative y,i G M> CiXCi , define the zero-sum 
game 7(2/*) with strategy sets Cm = C m := Ci and utilities 

u M {s l ,t i ) = -u m ( Si ,ti) := { ^ r f^ % J I* ~ ." 

otherwise. 

Lemma 2.15 ([7]). Fix a game T and consider T° . If y G A(C^J 7 then any strategy 
71 G Mm(7(yi)) x • • • x Mm(7(y„)) C A(C M ) satisfies UM(j,y) = 0. In particular v(T°) = 0, 
Ti is good against y, and A n (r) is good. 

Proof. Omitted. See [7] for a proof using the Minimax Theorem. □ 

Theorem 2.16 ([7]). For any game T, the value v(T°) = 0, so Mm(r°) = CE(r) and a 
correlated equilibrium ofT exists. 
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Proof. Combining Lemma 2.15 and Proposition 2.13, we get Mm(T ) = CE(T). For exis- 
tence, apply Proposition 2.8 to T with E = A n (r). □ 

This proof merits two remarks. First of all, since conv(A n (r)) = A(r), Proposition 2.8 
does not yield any benefit in this case over directly applying the Minimax Theorem to 
r°. Rather, we have used Proposition 2.8 to illustrate our proof strategy for Theorems 3.9 
and 4.7, in which we use stronger versions of Lemma 2.15 to choose £ with conv(S) C A(r). 

Second, note that in this case we know that there is a maximin strategy of T° in the good 
set A n (r): this is just the statement of Nash's Theorem. However, we cannot conclude this 
directly from the fact that A n (r) is a good set because of the remark after Proposition 2.8. 

2.3 Groups acting on games 

In this section we recall the notion of a group acting on a game, as defined by Nash [12]. 
All groups will be finite throughout. In any group e will denote the identity element. The 
subgroup generated by group elements gi, ■ ■ ■ ,g n will be denoted (gi, . . . , g n ). For n G N we 
will write Z n for the additive group of integers mod n and S n for the symmetric group on n 
letters. We will use cycle notation to express permutations. For example a — (1 2 3) (4 5) (6) 
is shorthand for 

o-(l) = 2, cr(2) = 3, tr(3) = 1, cr(4) = 5, cr(5) = 4, and <r(6) = 6. 

Definition 2.17. A left action of the group G on the set X is a map ■ : G x X — > X 

written with infix notation which satisfies the identity condition e-x = x and the associativity 
condition g ■ (h ■ x) = (gh) ■ x. A right action of G on X is a map • : X x G — > X such 
that x ■ e = x and (x ■ g) • h — x • (gh). 

We say that an action is linear if it extends to an action on an ambient vector space V 
containing X and the map x \— > x ■ g on V is linear for all g G G. An xeXis G-invariant 
if x ■ g = x for all g G G. The set of G-invariant elements is denoted Xq. 

Proposition 2.18. IfG acts linearly on the convex set X then there is a map avec : X — > Xq 
given by ave^rr) = ■nL ^2„ eG x ■ g. In particular if X is nonempty then X G is nonempty. 

Proof. For any x G X, avec(^) is a convex combination of elements x ■ g G X, hence 
avea{x) G X. For any h G G we have 



avec(a;) • h 



— ^x-g ■h=—^2(x-g)-h=—J2 x -(9h) 



\G\ 

1 ' geG 



\G\^ K *> \G\ 

1 ' g&G * * geG 



= TQi^2 x -9 = we G (x), 

1 ' geG 

where we have used linearity, the definition of a group action, and bijectivity of g i— >■ gh. D 



A left action of G on X induces right actions on many function spaces defined on X. 
For example IR X is the space of functions X — > R. For y G IR X we can define y ■ g G M x by 
(y ■ g)(x) = y(g -x). The condition that this is a right action of G on IR X follows immediately 
from the fact that we began with a left action of G on X . For finite X (the case of most 
interest to us), the same argument shows that G acts on A(X) on the right. 

Definition 2.19. We say that a group G acts on the game V if the following conditions 
hold. The group G acts on the left on the set of players / and |_JjC*> making g ■ Si G C g .i 
for Si G C{. Such actions automatically induce a left action of G on C = Y\ { Ci defined by 
(g ■ s) g .i = g ■ Si. We require that the utilities be invariant under the induced action on the 
right: u g .i ■ g = u iy i.e., u g .i(g ■ s) = Ui(s) for alH G /, s G C, and g G G. We say that G is a 
symmetry group of T and call elements of G symmetries of T. 

Note that an action of G on a game can be fully specified by its action on | |^ Ci or on C. 

One way to do this is to choose G to be a subgroup of the symmetric group on | |^ Ci or C 

satisfying the above properties. 

Definition 2.20. The stabilizer subgroup of player % is Gi := {g G G \ g ■ i = i}, and 

acts on Ci on the left. We say that the action of G is player trivial if Gi = G for all i, or 
in other words if g ■ i — i for all g and i. We say that the action of G is player transitive 
if for all i,j G I there exists g G G such that g ■ i = j. 

We illustrate the notion of group actions on a game using four examples. 

Example 2.21. Let T be any game and G any group. Define g ■ s = s for all g G G and s G C. 
This defines a player-trivial action of G on T called the trivial action. 

Example 2.22. A two-player finite game is often called a bimatrix game because it can be 
described by two matrices A and B, such that if player one plays strategy i and player two 
plays strategy j then their payoffs are A^ and Bij, respectively. If these matrices are square 
and B = A T then we call the game a symmetric bimatrix game. One example is the 
game of chicken, which has A = [| J] = B T . 

To put this in the context of group actions defined above, let each player's strategy set 
be C\ — Ci — {1, . . . , m} indexing the rows and columns of A and B. Define g ■ (i,j) = {j, i) 
for (i,j) G C, so g • (g • (i,j)) = (i,j)- The assumption B = A T is exactly the utility 
compatibility condition saying that this specifies an action of G = {e, g} = Z2 on this game. 
Of course, depending on the structure of A and B there may be other nontrivial symmetries 
as well. The element g swaps the players, so the action of G is player transitive. 

Example 2.23. Note that the condition that a bimatrix game be symmetric is not that 
A = A T and B = B T . Indeed, such a game need not have any nontrivial symmetries. 
For example, consider the game defined by A = [21] and B = [q?]- The unique Nash 
equilibrium of this game is for player 1 to play the mixed strategy p — [if] and player 2 
to play q — [ f § ]. Since the equilibrium is unique, any symmetry of the game must induce 
a corresponding symmetry of the equilibrium by Nash's Theorem. But the four entries of p 
and q are all distinct, so the only symmetry of this game is the trivial one. 
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(ui,u 2 ) 


H 2 


T 2 


Hi 


(1,-1) 


(-1,1) 


Ti 


(-1,1) 


(1,-1) 



Table 1: Matching pennies. Player 1 chooses rows and player 2 chooses columns. 

Example 2.24. Consider the game of matching pennies, whose utilities are shown in Table 1. 
The labels H and T stand for heads and tails, respectively, and the subscripts indicate 
the identities of the players for notational purposes. This a bimatrix game, but it is not a 
symmetric bimatrix game in the sense of Example 2.22. 

Nonetheless this game does have symmetries. The easiest to see is the map a which 

interchanges the roles of heads and tails. Letting g be the permutation of | |^ Cj given in 

cycle notation as g = (Hi Ti)(H 2 T 2 ), we define g ■ s« = g(si). Another symmetry is the 
permutation h = (Hi H 2 T\ T 2 ). These satisfy g 2 = e and h 2 = g, so G = (h) = Z4. Note 
that g acts on / as the identity whereas h swaps the players, so G acts player transitively, 
whereas (g) = Z 2 acts player trivially. 

Example 2.25. Now we consider an example of an n-player game with symmetries. Through- 
out this example all arithmetic will be done mod n. For simplicity in this example we will 
index the players using the members of Z n instead of the set {l,...,n}. Each player's 
strategy space will be Ci = Z n as well. Define 

, Jl, when Sj = Si_i + 1 

Ui(Si, . . . , s n ) = < . 

10, otherwise. 

Then we can define a symmetry g by g(s{) = Sj + 1, which increments each player's 
strategy by one mod n, but fixes the identities of the players. Clearly g is a permutation of 
order n. 

We can define another symmetry h which maps a strategy for player % to the same 
numbered strategy for player i + 1. That is to say, h acts on C by cyclically permuting its 
arguments. Again, h is a permutation of order n. Note that g and h commute, so together 
they generate a symmetry group G = Z n x Z n . Both (h) = Z n and G act player transitively, 
whereas (g) = Z n acts player trivially. If n is composite and factors as n = kl for k, I > 1 
then (h k ) = 7Li acts on T but neither player transitively nor player trivially. 

The left actions in the definition of a group action on a game induce linear right actions 
on function spaces such as A(T) C R c and A n (r) C R u * c \ The inclusion map R UiCi ->• R c 
is G-equivariant (commutes with the action of G), so with regard to this action it does not 
matter whether we choose to view A n (r) as a subset of R UiCi or of R . 

Because of the utility compatibility conditions of a group action on a game, the actions 
on A(r) and A n (T) restrict to actions on the sets CE(r) and NE(T), respectively. This 
allows us to define the G-invariant subsets Ag(T), A§(T), CEg(T), and NEg(T). The action 
of the stabilizer subgroup Gi on Ci allows us to define the G- invariant subset Ag^Cj). 
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The main theorem we set out to prove is the following. This theorem is most often 
applied in the case where G is the trivial group, but Nash proved the general case in [12] 
and so shall we. 

Nash's Theorem. A game with symmetry group G has a G-invariant Nash equilibrium. 

To prove this we will use Hart and Schmeidler's techniques in a new way. We will show 
that certain classes of symmetric games have correlated equilibria with a much higher degree 
of symmetry than might be expected without knowledge of Nash's Theorem. To illustrate 
what we mean, consider the following trivial improvement on Theorem 2.16. 

Proposition 2.26. A game with symmetry group G has a G-invariant correlated equilibrium. 

Proof. Apply Proposition 2.18 to a correlated equilibrium, which exists by Theorem 2.16. □ 

A priori we might not expect correlated equilibria with a greater degree of symmetry than 
predicted by Proposition 2.26 to exist. But viewing G-invariant Nash equilibria as correlated 
equilibria, we see that we can often guarantee much more. Suppose we have an n-player 
game which has identical strategy sets for all players and which is symmetric under cyclic 
permutations of the players, such as the game in Example 2.25. Then Proposition 2.26 yields 
a correlated equilibrium n which is invariant under cyclic permutations of the players, but 
need not be invariant under other permutations. On the other hand the Nash equilibrium 
p = (pi, . . . , p n ) given by Nash's Theorem satisfies p± — • • • — p n so the corresponding 
product distribution 7r(si, . . . , s n ) = pi(si) • • • pi(s n ) is a correlated equilibrium which is 
invariant under arbitrary permutations of the players. 

3 Exchangeable equilibria 

In this section we prove the existence of correlated equilibria with this higher degree of 
symmetry, as well as a useful factorization property, without appealing to Nash's Theorem. 

3.1 Exchangeable distributions 

First we need the notion of an exchangeable probability distribution. Our usage of the term 
"exchangeable" is slightly nonstandard but is closely related to the usual notion in the case 
when G acts player transitively. 

Definition 3.1. Viewing A§(r) as a nonconvex subset of the convex set Ag(T), we define 
the set of G-exchangeable probability distributions 

A£(r):=convAg(r)CA G (r). 

We use the term "exchangeable" because of the important case where the Cj are all 
equal and the group G acts player transitively (e.g. in Example 2.25). Then distributions 
in Aq(T) are invariant under arbitrary permutations of the players. Furthermore, by Dc 
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Finetti's Theorem 2 these are exactly the distributions which can be extended to exchangeable 
distributions on infinitely many copies of Ci, i.e., distributions invariant under permutations 
of finitely many indices. De Finetti's Theorem will not play a direct role in our analysis; 
here it merely serves to motivate Definition 3.1. 

To get a feel for these sets, we will look at them in the context of some examples. 

Example 2.21 (cont'd). Since G acts trivially we can ignore it entirely. Not all distributions 
are independent so A^(r) C Ac(r) = A(r), but Aq(T) = Aq(T). As we have seen, one 
inclusion is automatic. To prove the reverse note that for any s G C, S s — S SI ■ ■ ■ 5 Sn G 
A n (r) = A^(r). But for any it G A(r) we can write it = ^ seC n(s)S s , and such a convex 
combination of the S s is in Aq(T) by definition. 

Example 2.22 (cont'd) . For a symmetric bimatrix game T with m strategies per player, we can 
view probability distributions over C as m x m nonnegative matrices with entries summing 
to unity. The nontrivial symmetry g G G acts by swapping the players. From the definitions 
we see that Ag(r) consists of symmetric matrices and A§(r) of matrices which are outer 
products xx T for nonnegative column vectors x G M m . The elements of Aq(T) = conv A§(r) 
are exactly the (normalized) completely positive matrices studied in [1] . Clearly all such 
matrices are symmetric, elementwise nonnegative, and positive semidefinite; it turns out the 
converse holds if and only if m < 4 [3] . 

Example 2.24 (cont'd). The map on C induced by h is the permutation 

((i?i,H 2 ) (7\,H 2 ) {T^T 2 ) (H U T 2 )). 

In particular, a G-invariant probability distribution must assign equal probability to all 
four outcomes in C\ x Ci- There is only one such distribution and it is independent, so 

A£(r) = A£(r) = A G (r). 

Example 2.25 (cont'd). Recall that in this game there are n players and the Ci are the same 
for all i. The group G permutes the players cyclically. Therefore the elements of A^(r) are 
invariant under arbitrary permutations of the players, hence so are the elements of Aq(T). 
(The converse statement is false; that is to say, there are probability distributions over C 
which are invariant under arbitrary permutations of the players but are not in Aq(T). This 
is analogous to the presence in Example 2.22 of symmetric elementwise nonnegative matrices 
which are not positive semidefinite, hence not completely positive.) On the other hand, an 
element of Aq(T) need only be invariant under cyclic permutations of the players. 

We close the section on exchangeable distributions with a characterization of Aq(T) which 
we will not need until Section 4.3 but which logically belongs here. The characterization is 
a corollary of a more general convexity lemma which we state first. 

Lemma 3.2. Let f : K — > V be a continuous map from a compact set K to a finite- 
dimensional real vector space V . Extending f by linearity yields a weakly continuous map 



2 De Finetti's theorem states that the distribution of an infinite sequence of random variables X\, X2, ■ ■ . 
is invariant under permutations of finitely many of the random variables (exchangeable) if and only if it is 
a mixture of i.i.d. distributions [14]. 
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A(K) —¥ V given by n i— > J f dn which we also call f . The image of this map is f(A(K)) = 
conv(f(K)). 

Proof. The extension / is weakly continuous by definition. Clearly f(A(K)) is convex and 
contains f{K), so one containment is immediate. By linearity of integration, any linear 
inequality valid on f(K) must be valid on f(A(K)), so the reverse containment follows by a 
separating hyperplane argument (see Theorem 3.1.1 of [9] for details or Theorem 2.8 of [17] 
for an alternative topological argument). □ 

Corollary 3.3. The linear extension of the inclusion map A§(r) — > Ag(T) is weakly con- 
tinuous and maps A(Aj3,(T)) onto A^(T). 

3.2 Exchangeable equilibria 

We are now ready to define exchangeable equilibria. The proofs of the propositions in this 
section are direct algebraic manipulations and some are omitted. 

Definition 3.4. The set of G-exchangeable equilibria of a game T is 

XE G (r):=CE(r)nA£(r). 

When G can be inferred from context we simply refer to exchangeable equilibria. 

It is immediate from the definitions that conv(NEc(r)) C XEc(r) C CEg(T). There are 
examples in which all of these inclusions are strict [16], so proving non-emptiness of XEg(T) 
does not immediately prove non-emptiness of NEc(r). Nonetheless, this is an important 
step and the main result of this section. 

The proof that a G-exchangeable equilibrium exists proceeds along the same lines as the 
correlated equilibrium existence proof in Section 2.2. We again consider the zero-sum game 
r° and prove that a certain set is good in this game (Lemma 3.8). The difference is that the 
action of G yields a smaller good set, A§(r). To prove this lemma we need the following 
three symmetry results, which have straightforward proofs. 

Proposition 3.5. If G acts on T then G acts player trivially on T° by g ■ (s,(ri,ti)) = 
(g- s,(g-ri,g-U)). 

Proposition 3.6. If G acts player trivially on a zero-sum game, then a set £ C Ag(Cm) 
is good if and only if it is good against Aa{C m ). 

Proof. For all g G G, a G A g (Cm), and 6 G A{C m ) we have u M (a, 6 • g) — Um{o~ • g,9 • g) = 
u M (a, 9), so u M (a, 9) = u M (o; ave G (0)). □ 

Proposition 3.7. The map Mm(7(-)) is natural in the sense that if a : Cj —¥ Cj is a 
bisection and yi = yj o (a, a), then composition with a maps Mm(7(i/j)) to Mm(7(t/i)). 

Lemma 3.8. If G acts on the game T then the set Aq(T) is good in the zero-sum game T° 
of Definition 2. 12. 
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Proof. By Proposition 3.5 and Proposition 3.6, it suffices to consider only y G Ag(C° 
and show that there is a n G Aj3,(r) which is good against y. Lemma 2.15 states that any 
7r G S(y) := Mm(7(y 1 )) x • • • x Mm(j(y n )) C A n (r) is good against y. 

By Proposition 3.7 the action of G on A n (r) restricts to a linear action of G on S{y) 
since y G Ag(C^). Viewing S(y) as a convex subset of IR UiC, % Proposition 2.18 shows the 
G-invariant subset Sc(y) Q A|5(r) is nonempty, so A§(r) is good. D 

Theorem 3.9. A game with symmetry group G has a G- exchangeable equilibrium. 

Proof. By Theorem 2.16, Mm(r°) = CE(r). Lemma 3.8 shows we can apply Proposition 2.8 
to r° with E = Ag(r), proving that Mm(r°) n Ag(r) = XE G (r) is nonempty. D 

It is worth contrasting the proof that symmetric correlated equilibria exist (Proposi- 
tion 2.26) with this proof. Both involve averaging arguments to produce symmetric solutions. 
The difference is that in the proof of Proposition 2.26 the averaging occurs within the set 
A(r), whereas in the case of Theorem 3.9 (in particular Lemma 3.8), the averaging occurs 
within A n (r), viewed as a convex subset of R Ui \ By averaging within this smaller set, 
we guarantee that the resulting correlated equilibrium will have the additional symmetries 
discussed at the end of Section 2.3. 

The latter averaging argument requires a bit more care. In particular, Proposition 2.26 
is an immediate corollary of Theorem 2.16 on the existence of correlated equilibria. On 
the other hand, to prove Theorem 3.9 we have to "lift the hood" on Theorem 2.16 and use 
Lemma 2.15 on good sets. By doing so we exhibit a correlated equilibrium which we can 
prove lies in Aq(T) instead of just Ao(r). 

4 Higher order exchangeable equilibria 

In this section we begin with a game T and artificially add symmetries to produce two families 
of games ITT and HT with larger symmetry groups for each m G N. Having constructed 
these games, we can exploit our knowledge of their structure to improve Theorem 3.9 and 
show that there are distributions which are simultaneously exchangeable equilibria of both 
Il m r and S m r. We call such distributions order m exchangeable equilibria. 

We then use a compactness argument to exhibit a distribution which is simultaneously an 
order m exchangeable equilibrium for all m G N, called an order oo exchangeable equilibrium. 
We will see in the next section that for player-transitive symmetry groups, an order oo 
exchangeable equilibrium is just a mixture of symmetric Nash equilibria. 

Most of the work in this section consists of making the proper definitions. Once that is 
done, the proofs are rather short. 
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4.1 Powers of games 



To define order m G-exchangeable equilibria we will need two notions of a power of a game T. 
These are larger games in which multiple copies of T are played simultaneously 3 . Throughout 
this section we will take as fixed a game T with symmetry group G and a number m G N. 

Definition 4.1. For m G N, the m th power of T, denoted ITT, is a game in which m 
independent copies of T are played simultaneously. More specifically, HT has van players 
labeled by pairs i,j, 1 < i < n, 1 < j < m, strategy spaces H m Cij := Cj for all i,j with 



generic element s\, and utilities U m ua(s], . . . , s™) := Uiis 3 ,, s J , . . . . *■', 



H.jVl' 



'2? 



The contracted m th power of T, denoted S m r, is a game in which m copies of T 
are played simultaneously, but all by the same set of players. Specifically, S m r has n play- 
ers, strategy spaces S m Cj := C 4 m with generic element (s},..., sf 1 ) for all i, and utilities 
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Figure 1: Representing a 2-player game T with players choosing strategies s± and S2 as drawn 
on the left, the powers n"T and H"T are formed as shown. A shaded box represents actions 
controlled by a single player, whose utility is given by the sum over all interactions. 



Proposition 4.2. Let T be a game with symmetry group G and fix m G N. Then both powers 
n m r and HT are games with symmetry group G x S m and they satisfy: 



A 



Ct X On 



(ITT) = A 



Ct X Or? 



Ag x5m (ITT) C Ag x5 „ 
x 

Ct X O77 



A^ (n"T) C A x 



Cj X bri 






Proof. Both powers are invariant under arbitrary permutations of the copies and under 
symmetries in G applied to all of the copies simultaneously. In fact in the case of n"T we 
can apply a different symmetry in G to each copy independently so that Il m r is invariant 



3 These definitions can be generalized to define two possible products of games, so that the powers we 
define reduce to m-fold products of a game with itself. If we remove all mention of symmetry groups (so in 
particular exchangeable distributions and exchangeable equilibria are no longer defined) all the statements 
we make about Nash and correlated equilibria of these powers extend to corresponding statements about 
products with identical proofs. We will not need this level of generality, however, so to avoid complicating 
notation we focus on powers. 
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under the larger group G I S m (the wreath product of G and S m ), but we will not need this 
fact. 

Since G x S m acts on U m C and S m C in the same way, we get the first equality. The game 
n m r has more players than ST, so A^ xS (BT) has stronger independence conditions 
than A GxS (ST), yielding the strict containment. Taking convex hulls gives the final 
containment. □ 

Since A Gx>5m (BT) = A Gx> s m (ST), we can compare the conditions for a distribution 
it to be a correlated equilibrium of BT or ST. We use the notation and terminology 
introduced for Proposition 2.5 to do so. 

Proposition 4.3. Let (X^, . . . ,X™) be a random vector taking values in C m distributed 
according to vr G A Gx5m (BT) = A Gx5m (ST). Then 

• 71 is a correlated equilibrium of ITT if and only if X\ is a best response (in T) to 
f(X J _ i | Xf) almost surely for all i and j , and 

• tt is a correlated equilibrium of ST if and only if X\ is a best response (in T) to 
F(X J _i | X}, . . . , X™) almost surely for all i and j. 

Proof By Proposition 2.5, tt is a correlated equilibrium of BT if and only if X\ is a best 
response in I1T to P(X 1 , . . . , X j ~ l , X J _ i: X j+1 , ...,X m | X{) almost surely for all i and j. 
But the utility of player ij in LIT is Ui(X{, . . . , Xfy, so player ij can ignore X\ for all I ^ j. 
Now we consider when tt is a correlated equilibrium of S m r. Again by Proposition 2.5, 
this happens if and only if (Xl, . . . ,X™) is a best response in HT to P(Xl i; . . . ,X™ | 
X/, . . . , X™) almost surely for all i. The utility of player i in H"T is V • Ui(X{, X ] 2) . . . , X 3 n ) 

and no X\ appears in more than one term of this sum. Thus the sum is maximized when 
each term is maximized independently. That is to say ix G CE Gx 5 m (S m r) if and only if X\ 
is a best response in T to ¥{X 1 _ i \ X^, . . . , X™ - ) almost surely for all % and j. □ 

This characterization allows us to prove the following containments between equilibrium 
sets. One can construct examples showing that in general none of the inclusions in this 
proposition can be reversed. In particular, no containment holds between XE Gx s m (Il m r) 
and XE Gx 5 m (S m r) in either direction. This is connected to the fact that the inclusion 
between the sets of correlated equilibria of IPT and H"T goes in the opposite direction from 
the inclusion between the sets of Nash equilibria. 

Proposition 4.4. The equilibria o/TIT and ST satisfy 

NE Gx5m (ITT) C NE Gx5m (ST) C CE Gx5m (ST) C CE GxSm (BT) . 

Proof. We use Proposition 4.3. If X\ are distributed according to tt G A GxS (B m r) then the 
X\ are all independent, so the conditional distributions in Proposition 4.3 are equal to the 
corresponding unconditional distributions and both conditions are equivalent. This proves 
the first containment. 
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The second containment follows because Nash equilibria are always correlated equilibria. 
For the third containment, suppose X\ is a best response to f(X J _ i \ X}, . . . ,X™) almost 
surely. Summing over possible values of X~ 3 we get that X\ is a best response to F(X 3 _ i | X?) 
almost surely. D 

For any 1 < p < m we can define a projection map proj : Acxs^ITT) — > A Gx s p (lTT) 
which marginalizes out variables X p+1 , . . . ,X m . This map respects the structure of all the 
sets of distributions mentioned in Proposition 4.2, i.e., it restricts to maps A^ x5 (ITT) — > 
A Gx5p (ITT), A^ x5m (S m r) -» A^ xSp (E p T), etc. By Proposition 4.3 it also respect's the equi- 
librium structure of these games in the sense that XE Gx ,s m (]TT) maps into XE Gx ,s p (nT), 
NE GxS J~T) maps into NE Gx5p (HT), etc. 

One can show that if p — 1 then all of these maps are onto. We use this fact only to 
motivate what follows and not in any of the arguments below, so we omit its proof. In 
particular, this shows that proj (XE Gx5m (ITT)) = XE G (r) = proj(XE Gx <y m (ET)). One 
can give an example in which conv(NE G (r)) C XE G (r) [16], so neither of these projected 
sets of exchangeable equilibria approaches the convex hull of the Nash equilibria of T as m 
gets large. We will see that taking the intersection XE GxSm (ITT) fl XE Gx5m (E"T) fixes 
this problem. 

4.2 Order m G- exchangeable equilibria 

By Propositions 4.2 and 4.4 we have NE Gx5m (ITT) C XE Gx5m (ITT) n XE GxSm (E m T). 
Thus we expect the following definition not to be vacuous. 

Definition 4.5. The set of order m G-exchangeable equilibria of T is 

XE™(r) := XE GxSm (n m r) n XE Gx5m (s m r) , 

or equivalently by Propositions 4.2 and 4.4, 

XE™(r) := A^ xSm (U m T) n CE Gx5m (E m T) . 

The relationship between XE G l (r) and the sets of equilibria of the m th powers is summa- 
rized in Figure 2. We now prove order m G-exchangeable equilibria exist. 

Lemma 4.6. If G acts on the game Y then the set A GxS (ITT) is good in the zero-sum 
game (H"T)° of Definition 2.12. 

Proof. By Lemma 3.8, E := A^ x5 (HT) is good. The utilities in H"T are additively 
separable, so any mixed strategy a e S is payoff equivalent for the maximizer in (ST) to 
a mixed strategy a 1 e S' := A GxS (Il m r) given by the product of the marginals of a. We 
can apply Proposition 2.10 to E and E'. D 

Theorem 4.7. A game with symmetry group G has an order m G- exchangeable equilibrium 
for all m G N. 
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CE GxSm (IFT) < > CE GxSm (E m T) 



XE GxSm (iFT) 



XE Gx5m (~"T) 




XE™(r) 



NE, 



GxS r 



,(ITT) 



NE 



CrX On 



T) 



pro] 



-» 



CE G (r) 



XE G (r) 



proj(XE£(r)) 



NE G (r) 



Figure 2: At the left is a summary of the containments between equilibrium sets of the 
powers n m r and S"T proven in Section 4.1. An arrow A45 indicates 4C5. Under the 
marginalization map proj : A Gx s m (n m r) — > A G (T) each of these sets maps onto the set at 
the same height on the right. 



Proof. By Theorem 2.16, Mm((S m r)°) = CE(S m r). Lemma 4.6 shows we can apply Propo- 
sition 2.8 to (S"T)° with E = Ag^JIFT), so Mm((S"T)°) n Ag xSm (ITT) = XE™(r) is 
nonempty. □ 

For 1 < p < m, the marginalization map sends XE Gx 5 m (n m r) into XE Gx s (IPT) and 
XE GxSm (S m r) into XE Gx5p (S p r). Therefore it sends XE™(T) into XE G (r). Projecting the 
order m exchangeable equilibria into A G (r) for all m G N we obtain 

NE G (r) C conv(NE G (r)) C • • • C proj(XE G (r)) C proj(XE G (r)) c XE G (r) c CE G (r). 

This raises two natural questions: can we prove directly that f^^j^ proj(XE G l (r)) is non- 
empty? and does this intersection equal conv(NE G (r))? We will take up these two questions, 
respectively, in the following two sections. 

4.3 Order oo G-exchangeable equilibria 

Next we use a compactness argument to prove existence of an order oo G-exchangeable 
equilibrium, a distribution which is in some sense an order m G-exchangeable equilibrium 
for all finite m. As we have defined them the XE G l (r) are distributions over different numbers 
of copies of C, so they are not directly comparable and we can't just take their intersection. 
We could project them all into A G (r) and take the intersection there as mentioned above, 
but this would destroy some structure. Analytically it will be more convenient to view these 
sets within a larger space. 
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To take the intersection properly, we will transport all the XE^(T) into A(Aq(T)). Since 
S m acts transitively on the copies of the game in ITT, an element p G A^ x5 (Il m r) satisfies 
Pi = Pi f° r an hh an d k- Therefore the diagonal map Aj3,(T) — > A[J x5 (ITT) which sends 
a distribution to the product of m independent copies of itself is a homeomorphism. This 
extends to a homeomorphism A(Aq(T)) — > A(Aq xS (ITT)) of the corresponding spaces of 
distributions. If we compose this with the surjective map A(A^ x5 (ITT)) — > A^ xS (ITT) 
given by Corollary 3.3, we get a surjective map /i m : A(A G (T)) — > Aq xS (U m r). 

Define the inverse image sets A m = /i~ 1 (XE^(r)). Elements of A m are representations 
of order m G-exchangeable equilibria as mixtures of independent G-invariant distributions. 

Definition 4.8. The set of order oo G-exchangeable equilibria is XE^(r) := Hm=i Are- 

Theorem 4.9. A game with symmetry group G has an order oo G- exchangeable equilibrium. 

Proof. Each set XE^(r) is convex and closed by definition and nonempty by Theorem 4.7. 
The map [i m is linear, weakly continuous, has a compact Hausdorff domain, and is surjective. 
Therefore each A m is convex, compact, and nonempty. 

For 1 < p < m, the composition ji p o /i" 1 is a well-defined map Aq xS (IL m r) — > 
Aq xS (IFT) which coincides with the marginalization map discussed above. This map sends 
XE^(r) into XE^(r) as per the discussion at the end of Section 4.2, so the A m are nested 
A\ ^ A 2 5 A 3 D . . .. Thus they have nonempty, convex, compact intersection XE^ (T). □ 

5 Nash's Theorem 

5.1 The player-transitive case 

Theorem 5.1. If G acts player transitively on T, then XE^(r) = A(NEc(r)). 

Proof. If a G NE G (r) then p m (5 a ) G NE GxSm (ITT) C XEg(r), so 5 a G A m for all m and 
S ff G XEg'(r). But XE^(r) is convex and weakly closed, so A(NE G (T)) = conv{5 CT | a G 
NE G (r)}CXE£(r). 

For the converse let TZ be a random variable taking values in A^(r) distributed according 
to 7i G XE G 3 (r). Let X?, l<i<n,l<j< oo, be random variables taking values in Ci 
with distribution TZi which are conditionally independent given TZ. We must show that TZi 
is a best response to TZ^i almost surely. We will do this by approximating TZi an d TZ^i in 
terms of the X\. 

For each k G N the finite collection of random variables X\ with j < k is distributed 
according to fik{^) by construction; here we implicitly use the fact that pk{^) is an order 
oo G-exchangeable equilibrium, so Hki 71 ) G Aq xS (ITT). Furthermore we have Hki 71 ) G 
CE Gx £ fe (ET), so Proposition 4.3 states that for any 1 < j < k the strategy X\ is a best 
response to the random conditional distribution F(X 3 _ i | X}, . . . ,X!j°) almost surely. 

Since p k (n) is symmetric, P(x£. | X}, . . . ,X?) = P(X^ | X}, . . . ,Xf) for all i, j, and 
k. We define V\ to be this common random conditional distribution. Let y\ be the random 
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variable taking values in A(Cj) which is the empirical distribution of X}, . . . ,Xf. Then y{ 
is a best response to V\ whenever j < k. We will show that y\ and V\ converge to TZi and 
TZ-i, respectively, as j and k go to infinity. 

Let Ej be the completion of the a-algebra generated by X^,Xf, . . . and define V°° : = 
P(Xlj | Ej). Then V\ — >■ P°° almost surely as A; goes to infinity (Theorem 10.5.1 in [4]). 
Therefore y\ is a best response to "P°° for all j. By the strong law of large numbers, 
y\ converges almost surely to TZi as j goes to infinity, so TZi is a best response to V°° . 
Furthermore, TZi is measurable with respect to Ej because the y\ are. 

The X\ are conditionally independent given TZ, so we have V°° = E,(F(X^_ i \ 7Z) | Ej). 
Since G acts player transitively, for any player j we have TZj = TZi ■ g for some g G G, hence 
TZj is measurable with respect to Ej and so is TZ. In particular F{X\ i \ 7Z) is measurable 
with respect to Ej and we obtain 

P°° = E(P(Xlj | TZ) | Ej) = P(Xij | TZ) = TZ-i. 

This shows that TZi is a best response to 7£_j almost surely for all i, so TZ G NEg(T) almost 
surely and vr G A(NE G (r)). □ 

If G is the trivial group one can show that /ii(XE^ (T)) = CE(T) and /i 1 (A(NEc(r))) = 
conv(NE(r)). These sets are different for some games (e.g., chicken), so the above theorem 
can fail without the player-transitivity assumption. 

Nash's Theorem (player-transitive case). A game with player-transitive symmetry group 
G has a G-invariant Nash equilibrium. 

Proof. Combine Theorems 4.9 and 5.1, noting that A(0) = 0. □ 

5.2 Arbitrary symmetry groups 

In this section we show how to embed an arbitrary game T with symmetry group G in a 
game r Sym with a player-transitive symmetry group, preserving the existence of G-invariant 
Nash equilibria. This allows us to drop the player-transitivity assumption from the previous 
section, proving Nash's Theorem in full generality. 

There are a variety of ways to symmetrize games. The one we have chosen is a natural 
n-player generalization of von Neumann's tensor-sum symmetrization discussed in [6]. The 
idea is that each of the n players in r Sym plays all the roles of the players in T simultaneously. 
The players in r Sym play n\ copies of T, one for each assignment of players in r Sym to roles 
in T. A player's utility in r Sym is the sum of his utilities over the copies. 

Definition 5.2. Given an n-player game T with strategy sets Ci and utilities Ui we define its 
symmetrization r Sym to be the n-player game with strategy sets Cj ym := C (with typical 
strategy s l — (s\, . . . , s l n )) and utilities 



uf ym { 



s ) '■= "^2 u T{ i) (d(r * s)) , 
res„ 
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where s = (s\ . . . , s n ) E C Sym = C n , * : S n x C Sym ->■ C Sym is defined by (r * s) k := s T *(*>, 
and d : C Sym ^ C is defined by [d(s)] fc := s£. 

We now show that r Sym is a game with player-transitive symmetry group. We will use * 
to denote the action on r Sym to distinguish it from the action • on I\ 

Proposition 5.3. If F is a game with symmetry group G then r Sym is a game with player- 
transitive symmetry group G x S n , where a E S n acts by * as defined above and g E G acts 
by 

0*(s\...,s n ) ^(g-s\...,g-s n ). 

Proof. Note that * defines an action of G on C Sym . Also, for a,r E S n we have 

(r * (a * s )) fe = (a * s) T_1 « = s-" 1 ^- 1 ^)) = s (T<r)-i( fc ) = ((r(T) * s) fe ? 

so * is an action of S n on C Sym as well. These actions commute, so together they define 
an action * of G x S n on C ym . Note that the induced actions on players are g *i = i and 
a*i — cr(i). 

To show that this is an action of G x S n on r Sym it suffices to show that the utilities of 
pSym are invariant under the action of any a E S n and any g E C To see the former, let 
a £ S n . Then we have 

W S*V * s ) = Y u T(°(.i)) ( d ( r * ( a * s ))) = Y u (.™m {d{{r&) ~ ks )=Y u ^) ( rf ( r * s )) 

reS n reS n T(zS„ 

= «?"», 

where we have used in the penultimate equation the fact that S n is a group, so the map 
r I—)- ro" is a bijection. To see the latter, let g E G and let 7 E S n be the permutation induced 
by g on the set of players in T. Then we have d(g * s) = g ■ d{^ 1 * s), so 



W ST(0 * s ) = $^ M r(i) (d(r * (# * s))) = ^ w T(i) (d(^ -k (r • s))) 

= J^ «r(i) (5 • d(7 _1 * (T * S))) = Y u (^r){i) (d((7"V) * s)) 

T"G5 n tGS„ 

= Y u ^ ( d ( r * s )) = ^ Sym ( s )> 

t£S„ 

where the fourth equation follows because g is a symmetry of I\ Clearly S n acts transitively 
on the set of players. □ 

Nash's Theorem. A game with symmetry group G has a G-invariant Nash equilibrium. 

Proof. Let T be a game with symmetry group G. Then r Sym is a game with player-transitive 
symmetry group G x S n by Proposition 5.3, so it has a (G x S' n )-symmetric Nash equilibrium 
by the player-transitive version of Nash's Theorem. By definition of the action of G X S n 
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on r Sym , this Nash equilibrium is of the form (p, ...,p), with p G Aa(T). Notice that 
for each player i, each utility u k ym (s 1 , . . . , s n ) is a sum of functions which only depend 
on s l j for a single value of j. Thus p is payoff equivalent to the product of its marginals 
Pi x • • • x p n G A^(r). Therefore we can take the Nash equilibrium (p, . . . ,p) to be such 
that p G Ag(r) by Proposition 2.11. 

It remains to verify that p G NEc(r). For any s l G C we can compute 

^4 7m (p, ...,p,S l ,p,...,p)= ^2 U r(i) (pl, ■■■, Pr(i)-1, 4(i). Pr(i)+1. • • • , Pn) 

rSS n 

n 

= (n- l)\^Uj(p u . . . , Pj_i, s), Pj+u ■ ■ -,Pn)- 
3=1 

For each value of j we can vary the s* component of s l independently and it is a best response 
for player % to play p in r Sym if the rest of the players play p, so we must have 

Uj(Pi, • • • , Pj-i, Sj, p j+1 , ...,p n )< Uj{p) 
for all players j and all s^ G Cj, i.e., p G NE G (r). D 

6 Conclusion 

We have shown that by studying group actions on games and introducing the notion of 
exchangeable equilibrium, we can prove Nash's Theorem. To the authors' knowledge, this 
is the first proof of this theorem which uses convexity-based methods (i.e., the minimax 
theorem). Previous proofs use path-following arguments or fixed-point theorems, which are 
essentially equivalent to path-following arguments by Sperner's Lemma. 

This new proof invites new approaches for computing or approximating Nash equilibria. 
One can rewrite the existence proof above for (order m) exchangeable equilibria in terms 
of linear programs and separation arguments instead of the Minimax Theorem and apply 
the ellipsoid algorithm, just as Papadimitriou has done for Hart and Schmeidler's proof of 
the existence of correlated equilibria [13]. This shows that exchangeable equilibria can be 
computed in polynomial time, at least under some assumptions on the parameters. For 
example, order m exchangeable equilibria of symmetric bimatrix games can be computed in 
polynomial time for fixed m. 

We have seen that in the player-transitive case order m exchangeable equilibria converge 
to convex combinations of Nash equilibria as m goes to infinity. There are a variety of 
ways one could imagine "rounding" exchangeable equilibria to try to produce approximate 
Nash equilibria. We leave the analysis of such procedures, along with the question of which 
assumptions on G allow computation of exchangeable equilibria in polynomial time, for 
future work. 

The power of these methods suggests that exchangeable equilibria should not merely 
be viewed as a step on the way to Nash equilibria. Rather, they deserve further study 
in their own right. We consider several interpretations of exchangeable equilibria and the 
applications they suggest in [16]. 
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